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ABSTRACT 

Reasoning about degrees of belief has been investigated in 
the past by a number of authors and has a number of prac- 
tical applications in real life. In this paper we present a 
unified framework to model and verify degrees of belief in 
a system of agents. In particular, we describe an exten- 
sion of the temporal-epistemic logic CTLK and we intro- 
duce a semantics based on interpreted systems for this ex- 
tension. In this way, degrees of beliefs do not need to be 
provided externally, but can be derived automatically from 
the possible executions of the system, thereby providing a 
computationally grounded formalism. We leverage the se- 
mantics to (a) construct a model checking algorithm, (b) 
investigate its complexity, (c) provide a Java implementa- 
tion of the model checking algorithm, and (d) evaluate our 
approach using the standard benchmark of the dining cryp- 
tographers. Finally, we provide a detailed case study: using 
our framework and our implementation, we assess and ver- 
ify the situational awareness of the pilot of Air France 447 
flying in off-nominal conditions. 

1. INTRODUCTION 

Suppose you draw a seven of diamonds from a deck of 
cards, and your friend Alice draws another card that she 
keeps secret. You obviously know that you have a seven 
of diamonds, and you obviously do not know Alice’s card. 
However, you can believe that Alice has an ace of spades 
(nothing rules out this possibility). You can also believe 
that Alice has a card whose suite is hearts. It also seems 
natural to think that the latter belief has a “greater weight” 
than the former or, equivalently, that you have a “greater 
degree of belief” in the latter. 

A standard approach to belief quantification involves the 
use of probabilities and the example of cards described above 
is interpreted in terms of probabilities by almost all read- 
ers. However, beliefs can be quantified using a number of 
other approaches (see [13] for a detailed overview). One 
way to characterise this literature is by referring to objective 
and subjective assignments to degrees of belief. Subjective 
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assignments differentiate between actual probabilities and 
agents’ beliefs, while objective assignments refer to actual 
features in the real world (for instance when modelling a bi- 
ased coin). In this paper we employ the term degrees of be- 
lief and we avoid references to probabilities, thereby taking 
what could seem a subjective approach. Nonetheless, there 
is a connection between our approach to modelling degrees 
of belief and probability distributions; this link will become 
clear after the introduction of our technical machinery and 
we will return to this connection in Section 6. For the time 
being, however, we ask the reader to avoid interpreting the 
weight of doxastic modalities in terms of probabilities, as 
our aim here is to introduce a unified framework to model 
and verify degrees of belief in a system of agents. More in 
detail, our contributions can be summarised as follows: 

• We provide a computationally grounded formalism to 
reason about degrees of belief by introducing an ex- 
tension of the logic CTLK whose semantics is based 
on interpreted systems [9]. We name this extension 
COGWED: a Computationally Grounded, wEighter 
Doxastic Logic. 

• We introduce a model checking algorithm for COG- 
WED by extending the standard algorithm for CTLK, 
together with its complexity analysis. 

• We implement and we release as open source a model 
checker for COGWED and we use the benchmark of 
the dining cryptographers to prove the feasibility of 
our approach. 

• We employ our model checker to verify the key prop- 
erties of a safety-critical scenario. 

The rest of the paper is organised as follows: in Section 2 
we review the formalism of interpreted systems; in Section 3 
we present COGWED and its model checking algorithm; in 
Section 4 we introduce a model checker, its implementation 
and its performance evaluation on the protocol of the dining 
cryptographers. Finally, in Section 5 we introduce a moti- 
vational example where we show how our approach can be 
used to characterise the situational awareness of a pilot fly- 
ing in off-nominal conditions. In particular, we consider the 
model of the Air France 447 accident provided in [1] and 
we evaluate the situational awareness of the pilot when a 
stall occurs. We show that there exist cases in this model in 
which the plane in actually stalling, but the pilot has a very 
low degree of belief about the stall, a situation than can be 
formally analysed with our tool. 


2. PRELIMINARIES 
2.1 Interpreted Systems 

We employ here the formalism of Interpreted Systems from [9] 
to describe a system of agents. 

In particular, IS = (G, Rt, V) where 

• G = X I; is a finite set of global states, obtained as 

1 ••■n 

the cartesian product of n sets of local states (one set 
for each agent); 

• Rt £ G x G is a temporal relation (it is assumed that 
each state has at least a successor); 

• V : AP — * 2 G is an evaluation function for atomic 
propositions; 


The formalism of interpreted systems presented in [9] and 
employed in other model checkers such as [18, 12] also in- 
cludes the notions of agents’ actions and agents’ protocols: 
to keep our presentation simple, we do not consider these 
here, as they play no role in the semantics for the logic pre- 
sented below. 

We define a set of n equivalence relations (one for each 
agent): let g = (h, . . . , l n ) and g' = (l[, be two global 

states from G; we define gRtg' iff h = l'i, i.e., two global 
states g , g' are equivalent for agent i iff the local state of 
agent i is the same in g and in g' (notice that these are the 
standard epistemic relations used in [9] to interpret epis- 
temic modalities). We define { g }_R i to be the equivalence 
class of the global state g w.r.t. Rt. 

Given an interpreted system IS and a global state g, logic 
formulas involving CTL and epistemic operators can be in- 
terpreted as follows (we refer to [9] and references therein 
for more details about CTL syntax and semantics): 

9 e V(p); 

is,g¥ V, 

IS,g |= ip 
and IS, g \ = ip\ 
there exists g' e G s.t. gRtg' 
and IS,g' \= p; 
there exists g' e G s.t. gRtg' 
and IS,g’ \= p; 

there exists a path ir = (g,gi, . . .) 
such that, for all i, giR t gt+ i 
and IS,gi \= <p; 

there exists a path n = (g, gi, . . . ) 
and an index j such that IS,gj |= i/’ 
and IS, gt \ = ip for all i ^ j; 
gRig ' implies IS, g |= p. 

With slight abuse of notation we denote with V(p) the 
set of states of an interpreted system IS in which ip holds. 
This logic is usually named CTLK and can include group 
epistemic modalities to reason about distributed and com- 
mon knowledge. In the next section we will extend this logic 
with doxastic operators. 


IS,g\=p 

iff 

IS,g |= -'P 

iff 

IS, g |= p a %!) 

iff 

IS, g [= EXp 

iff 

IS,g \= EXp 

iff 

IS,g \= EGp 

iff 

IS,g \= E[pUif>] 

iff 


iff 


2.2 Model checking Interpreted Systems 

Given a logic formula p and an appropriate model M for 
p, in general terms model checking is the problem of estab- 
lishing whether or not M \= p, usually in an automated way. 
In the context of Interpreted Systems, model checking is the 
problem of verifying that a given CTLK formula p holds in 
all the global states of an Interpreted System IS. 


The complexity of model checking CTLK formulae in a 
given Interpreted System is polynomial in the size of the 
model and formula [6, 9]. The standard algorithm operates 
recursively on the structure of the formula by “labelling” 
the global states of the Interpreted System with the sub- 
formulae that are true there. We refer to [6, 9] for additional 
details. 

We remark here that, in many cases, the model is gen- 
erated from a succinct description by means of model vari- 
ables; in this case, adding a simple Boolean variable causes 
the model to double in size. This is known as the state 
explosion problem and the complexity of model checking 
CTLK formulae against succinct representations requires de- 
terministic algorithms that have an exponential complexity 
in the size of the representation [19]. Symbolic algorithms 
using Ordered Binary Decision Diagrams and reduction to 
SAT problems have been successfully employed in various 
tools [18, 12] for multi-agent system verification to tackle 
this complexity. We return to this issue in Section 3.3. 

3. MODEL CHECKING COGWED 

In this section we introduce the syntax of COGWED, its 
semantics with some key equivalences and a model checking 
algorithm for it. We also present some complexity consider- 
ations. 


3.1 COGWED Syntax and Semantics 

Let ~ be one of the following comparison operators: {< 
, =, >}. The syntax of COGWED is as follows: 

p ::= p \ 'p | p a ip | B\ x p | EXp \ EGp \ E[pUp] \ Kip 

Where: 


• p is an atomic proposition from a set AP- 

• i is an index for agents, ranging from 1 to n; 

• x is a real number, 0 < x < 1; 

• EXp, EGp, E[pUtl>] are standard CTL temporal op- 
erators. 

• Ki is the standard epistemic operator. 


Essentially, COGWED extends CTLK with the additional 
operators B' (one for each agent) and with comparison op- 
erators. The formula B~ x p is read as “Agent i believes p 
with a degree of belief ~ x. For instance, B^ 0 2 {p v q) is 
read as “Agent 2 believes (p v q) with a degree of belief less 
or equal than 0.2, and 5 (B^ 0 .i(p)) means that “Agent 
2 believes with degree exactly equal to 0.5 that Agent 1 be- 
lieves with degree at most 0.1 that p” (where p could mean 
“agent 2 has an ace of space)”. As we will see below, B l =1 p 
is equivalent to Kip. COGWED formulae are evaluated in 
Interpreted Systems by extending the definitions provided 
in the previous section with the following: 

tc l os -a I 

IS, g |= B^ x p iff — Kg}H j ~ x 

The intuition behind this definition is the following: the 
degree of belief that an agent associates to a formula p in 
a global state g is the ratio between the number of states 
of {g}i (the equivalence class of g) in which p is true and 
the total number of states in {g}i. For instance, considering 
again the scenario in which you draw a seven of diamonds 
from a deck of card, and Alice draws another card that she 
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// We associate a set of equiv alence 
/ / classes to each agent : 

Map <Integer , Set <Set <Gstate>» rk ; 

// This method computes the set of 

// states in which B^ x f is true 

public Set<Gstate> satB(int i , Formula f , 

String op , float x) { 
Set<Gstate> previous = SAT( f ) ; 

Set<Gstate> result = new Set(); 

for (Set<Gstate> eqClass : rk.get(i)) { 

• n ( | eqClass n previous | \ r 

1 1 V [eqCTaii] ~ X ) 1 

result . add ( eqClass ) ; 

} 

} 

return result ; 


Figure 1: Java-style algorithm sketch 


keeps secret. If the deck has 52 cards overall, your belief 
about the fact that Alice has an ace of diamonds has a degree 
of 1/51, and your belief that Alice has a card whose suite 
is hearts has a degree of 13/51. As a result, the following 
formula is true: 

^2ao5 WC1 (Alice_ace_spade) A B^o V 2 ewer (Alice_hearts) 

This definition of degrees of beliefs is computationally groun- 
ded in the sense of Wooldridge [23]: modalities are inter- 
preted directly on the set of possible computations of a 
multi-agent system (equivalently: modalities are interpreted 
on a Kripke model that corresponds to the possible compu- 
tations of a multi-agent systems), and there is no need to 
provide weights as part of the model. We refer to Section 6 
for a comparison with other existing approaches to evaluate 
degrees of belief. 

The following formulas are valid in all COGWED models 
as a result of simple arithmetic considerations: 

1. B l ^ x ip — > B^yip for all y ^ x\ 

2. B^ x ip — * B^ y ip for all y ^ x; 

3 - B^. x ip «-> B l ^ 1 _ x - > —'ip 

Finally, it is easy to see that Bh i<p is equivalent to Kup, 
i.e., a degree of belief equal to 1 corresponds to the standard 
epistemic operator. Dually, as a result of the third formula 
above, it is also true that B= 0 tp <-> 

3.2 The algorithm 

In this section we describe a model checking algorithm for 
the operator B\ x f . We do this by describing a method satB 
that can be included in the standard model checking algo- 
rithm for CTLK. A Java-like description of the algorithm is 
provided in Figure 1. 

The method employs the set of equivalence classes for each 
agent; this set can be computed by partitioning the set of 
global states (remember that each global state is a tuple of 
local states). The result of this operation is the map rk 
(line 3), which associates an agent ID (in the form of an 
Integer variable) to a set of sets of global states (i.e., the set 
of equivalence classes). 

The method satB returns the set of global states satisfy- 
ing the formula B~ x f. It starts by (recursively) calling a 


method SAT(f) that computes the set of states in which the 
formula / is true (line 9). Then, it iterates over the equiv- 
alence classes of agent i (line 11). In line 12 the method 
computes the ratio of the set in which the formula is true in 
a given equivalence class over the size of the actual equiv- 
alence class. It this ratio satisfies the appropriate relation 
then the method adds the whole equivalence class to the 
set of states in which the formula is true (line 13). The in- 
tersection of sets of states can be performed with standard 
library functions provided by Java; we refer to the source 
code available online for additional details about the actual 
implementation. The final result is returned at line 16. 

As mentioned above, notice that the algorithm does not 
operate on individual states. Instead, once the equivalence 
classes are built, the algorithm works with sets of states. 
We investigate the complexity of this algorithm in the next 
section. 

3.3 Complexity considerations 

Model checking CTLK formulae in an interpreted system 
takes time polynomial in the size of the formula and in the 
size of the model [9]. All the operations in the algorithm de- 
scribed in Figure 1 require at most polynomial time: com- 
puting the set of equivalence classes, iterating over them, 
and computing intersection of states. Therefore, the method 
described above remains in the same polynomial complexity 
class of the standard CTLK model checking algorithm. 

As mentioned in Section 2.2, in practical applications the 
actual state space is likely to explode as a result of the num- 
ber of variables employed to model a given scenario. A num- 
ber of techniques are available to manage large state spaces. 
In particular, Ordered Binary Decision Diagrams (OBDDs) 
are employed in model checkers for multi-agent systems such 
as MCMAS [18] and MCK [12]. The algorithm of Figure 1 
operates on set of states from line 7 to line 17 and only 
performs intersections of sets: these operations can be per- 
formed on the OBDDs for the sets of states, and therefore 
this part of the algorithm can be executed symbolically. The 
computation of equivalence classes needed at line 3, how- 
ever, may require in the worst case the explicit enumeration 
of all reachable states, if all global states are epistemically 
different for a given agent. This is rarely the case and, in 
fact, the number of equivalence classes is normally orders of 
magnitude smaller than the number of global states. This 
is indeed the case in the examples that we present below in 
Section 4.2 and in Section 5. 


4. MC-COGWED: A TOOL TO VERIFY COG- 
WED PROPERTIES 

In this section we describe a model checker for the verifi- 
cation of COGWED properties, called Mc-COGWED. This 
is a prototype implementation that is used to evaluate the 
algorithm presented above on the standard example of the 
dining cryptographers. All the source code, the benchmarks 
of the dining cryptographers and the card examples, and 
a pre-compiled version are available from this link: https : 

/ /sites . google . com/site/mccogwed/ 

4.1 Implementation overview 

Mc-COGWED is implemented entirely in Java. The in- 
put language of the model checker is a simple description of 
the states and transitions in a model. An example of this 
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// 

Jv 

st two agents 


N— 

2; 



// 

Th 

e list of global states 


SI 

= 

(cl , c2 ) ; 


S2 

= 

(cl , c3 ) ; 


S3 

= 

(c2 , cl ) ; 


S4 

= 

(c2 , c3 ) ; 


S5 

= 

( c3 , cl ) ; 


S6 

= 

(c3 , c2 ) ; 


// 

If 

needed, a temporal rel 

ation 

// 

can be specified using th 

e following 

// 

RT = { (SI , S2) , ( SI , S3 ) , . . 

■}; 

// 

Th 

e labelling fu notion : 


ag 

ent 

l_has_cardl = { SI , S2 

}; 

ag 

ent 

2_has_cardl = { S3, S5 

}; 

// 

[■ 



Figure 2: Input file for Mc-COGWED (3 cards) 



0 500 1000 1500 2000 2500 

N. cards 


Figure 3: Performance results for the card example. 


language is provided in Figure 2: this example describes a 
scenario with 2 agents and all the possible states resulting 
from the agents picking a card from a deck with three cards. 
More in detail: in line 2 we specify the number of agents. 
The lines from 5 to 10 encode the set G of global states; 
each global state is identified by an ID (SI to S6) and is 
described by a pair of local states as there are only 2 agents 
in this example. For instance, S3 = (c2 , cl) corresponds to 
the global state in which the local state for the first agent 
is c2 and the local state for the second agent is cl, i.e. , the 
global state in which the first agent has card 2 and the first 
agent has card 1. We do not include a temporal relation 
for this simple example but, as exemplified in the comment 
at line 14, the temporal relation is represented by a list of 
pairs of states (the protocol of the dining cryptographers be- 
low contains a temporal relation). Finally, lines 17 and 18 
provide an example definition of two atomic propositions. 
The first atomic proposition is true when agent 1 has card 
1 (in global states SI and S2), while the second proposition 
is true when agent 2 has card 1 (in global states S3 and S5). 

This is a very simple language, but our aim here is to pro- 
vide a concrete assessment of the complexity of the approach 
and to show that COGWED can be successfully employed 
in the verification of real-life scenarios (see Section 5). 

Mc-COGWED parses the input file using ANTLR [20] and 
builds an explicit representation of the model using standard 
Java structures for sets and maps. The epistemic relations 
are automatically generated from the structure of the global 
states by imposing the equivalence of local states. 

In addition to the input file for the model, Mc-COGWED 
takes a COGWED formula as an input parameter from the 
command line. We have implemented the model checking 
algorithm for the minimal set of temporal operators EX, EG 
and EU, for Boolean expressions and for the belief operator 
B~ x . Mc-COGWED operates recursively on the structure 
of the formula and generates a set of (global) states that 
can be either explicitly printed on screen, or the tool could 
simply report the number of global states where the input 
formula is true. 

We provide a generator for the card example mentioned 
above in the directory examples/ of the source code. The 
generator takes as an input parameter the number of cards 
to be generated and creates an input file similar to the one in 


Figure 2. We report in Figure 3 the number of global states 
(right scale) and the execution time (left scale) for a number 
of cards ranging from 100 to 2500 for the verification of the 
formula agent lJhas_cardl — » (K 1 (B 2 i agentl_has_cardl)). 

(n-l) 

This formula expresses the fact that, if agent 1 has card 1 
from a deck of n cards, then agent 1 knows that agent 2 be- 
lieves with a degree of belief less than that agent 1 has 
card 1: this formula is true in all the states of the model and 
forces the exploration of all the equivalence classes. Figure 3 
shows that the tool is able to verify up to 2500 cards (corre- 
sponding to a state space of approximately 7 million states) 
in less than 45 minutes. The figure displays the quadratic 
dependance of the total number of global states on the num- 
ber of cards, and a polynomial of higher degree for the ver- 
ification time: this is due to the two nested epistemic and 
doxastic modalities together with the computation of two 
set intersections. These and all the results below are ob- 
tained on a Macbook Pro, 2.4 GHz Intel Core i7 with 16 
GB RAM and running Mac OS X 10.8.5. Mc-COGWED 
was compiled using Java version 1.7.0, revision 40, and the 
Java virtual machine was configured with a heap size of 12 
GB. A more detailed performance evaluation is carried out 
in the next section. 

4.2 Performance evaluation: the dining cryp- 
tographers 

In this section we conduct a more detailed evaluation of 
performance for Mc-COGWED using the protocol of the 
dining cryptographers. The protocol of the dining cryptog- 
raphers is a standard benchmark in the multi-agent verifica- 
tion community, as it employs temporal-epistemic specifica- 
tions and it can be easily scaled up. The protocol, originally 
described in [4], is exemplified by the following scenario: 
three cryptographers sit at a round table at a restaurant. A 
waiter informs them that the bill has already been paid for. 

The cryptographers now wonder whether one of them paid 
for the bill, or whether it was paid by their company. To 
preserve the anonymity of the payer, they run the following 
protocol: each one of them flips a coin behind a menu on 
their right, so that this coin is only visible by the person 
who flipped the coin and by the next cryptographer to the 


right. In this way, each cryptographer sees two coins. After 
the initial round of coin tosses, each cryptographer has to 
announce whether s/he sees two equal coins (e.g, two heads 
or two tails), or two different coins. However, if the cryptog- 
rapher paid for the dinner, than s/he has to say the opposite 
of what s/he sees. The key property of the protocol is that, if 
there is an even number of cryptographers announcing that 
the coins are different, then the company paid for the din- 
ner; if the number of “different” utterances is odd, however, 
then someone at the table paid for dinner. In this case, it is 
possible to verify the key epistemic property: 

(odd a ^paidj) — > (A 1 (paid 2 v paid 3 ) a — 'A 1 (paid 2 ) a — 'it’ 1 (paid 3 )) 
which encodes the fact that, if the first cryptographer did 
not pay for the dinner and there is an odd number of “differ- 
ent” utterances, then the first cryptographer knows that ei- 
ther cryptographer 2 or cryptographer 3 paid for the dinner 
(i.e., the cryptographer knows the disjunction), but cryp- 
tographer 1 does not know that cryptographer 2 paid, nor 
cryptographer 3. It is also possible to verify that the same 
formula holds for any number of cryptographers greater than 
2. In COGWED we can refine this formula and introduce a 
degree of belief for the first cryptographer, in the case s/he 
did not pay: 


AG 


^(odd A — 'paidj) 



This formula captures the fact that an odd number of utter- 
ances places an equal degree of belief on the fact that any 
of the remaining cryptographers could be the payer. 

We have implemented a Java generator for the dining 
cryptographers in Mc-COGWED; this generator is available 
under examples/ in the source files and takes the number 
of cryptographers as an input parameter. Each cryptogra- 
pher is modelled with 4 local variables: value of left and 
right coin (possible values: Empty, Head, Tail), whether the 
cryptographer is the payer (Empty, Yes, No), and the parity 
of “different” utterances (Empty, Even, Odd). In the ini- 
tial state the value of these variables is set to empty for all 
cryptographers. The generator then runs the protocol by 
producing a random initial configuration and outputs a file 
in COGWED format with the set of reachable global states, 
the temporal transition relation for these states, and an ap- 
propriate labelling function for the global states. This file 
is then passed to Mc-COGWED, together with the formula 
described above. 

We ran experiments with a number of cryptographers rang- 
ing from 3 to 15. Experimental results are reported in Ta- 
ble 1. The first column reports the number of cryptogra- 
phers; the second column labelled with IS) reports the size of 
the state space (in our encoding of the example this is simply 
(3 4 ) n , where n is the number of cryptographers); the third 
column |G| is the number of reachable states as computed 
by our generator; the fourth column R t reports the number 
of pairs in the transition relation; the fifth column reports 
the time required to generate the set of reachable states and 
to write this set to a file. The final column reports the time 
required to parse this file and verify the formula reported 
above by the actual Mc-COGWED model checker. The size 
of the generated file exceeds 300 Mb for 16 cryptographers 
and this causes the ANTLR parser to run out of memory 
before invoking the generation of epistemic relations and 
the verification of the formula. In all the other cases, the 
overall execution time obtained by adding the generation of 


N 

[si 

|G| 

m 

gen. 

time (s) 

verif. 
time (s) 

3 

5 ■ 10 5 

65 

96 

0.11 

0.12 

4 

4 • 10® 

161 

240 

0.12 

0.16 

5 

3 • 10 s 

385 

576 

0.15 

0.31 

6 

2 ■ 10 10 

897 

1344 

0.18 

0.33 

7 

2 ■ 10 12 

2049 

3072 

0.25 

0.46 

8 

1.15 • 10 15 

4609 

6912 

0.38 

0.49 

9 

1.50 • 10 17 

10241 

15360 

0.51 

0.83 

10 

1.22 • 10 19 

22529 

33792 

0.67 

1.27 

11 

9.85 • 10 2 ° 

49153 

73728 

1.17 

4.16 

12 

7.98 • 10 22 

106497 

159744 

1.94 

6.74 

13 

6.46 • 10 24 

229377 

344064 

3.35 

23.48 

14 

5.23 • 10 26 

491521 

737280 

6.77 

70.38 

15 
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Table 1: Dining cryptographers: results 


the reachable state space and the actual verification time 
remains below 4 minutes even for 15 cryptographers; we 
are therefore confident that a more compact representation 
of the example using a more expressive modelling language 
for COGWED models could enable the verification of even 
larger state spaces. 

These results are very encouraging, as they are compara- 
ble to what highly optimised and symbolic model checkers 
such as MCMAS and MCK can achieve for standard epis- 
temic modalities (see, for instance, the results reported in 
Table 2 of [18]). Our results thus show that reasoning about 
degrees of beliefs in a system of agents is feasible even for 
large state spaces, even for formulae involving both temporal 
and doxastic modalities. 

Besides being computationally tractable, in the next sec- 
tion we show how model checking COGWED can have prac- 
tical applications in analysing safety-critical scenarios. 


5. CASE STUDY: THE AIR FRANCE 447 AC- 
CIDENT 

In the previous sections we have employed COGWED to 
characterise two scenarios that are typical in security and 
communication protocols. However, degrees of belief can be 
used to reason formally about other specification patterns. 
In this section we show how situational awareness can be 
assessed using COGWED. Informally, situational awareness 
is the ability of an agent (typically human) to assess a situ- 
ation and to understand how the environment will react to 
the agent’s actions. Situational awareness is a key factor for 
decision makers in safety-critical situations, such as airplane 
pilots, medical doctors, firemen, etc, and it has been inves- 
tigated extensively in the past a number or research areas, 
including psychology [10]. Here we focus on the aeronautic 
domain, working in collaboration with domain experts from 
NASA Ames. 

5.1 A model for AF447 

The Air France flight 447 from Rio de Janeiro to Paris 
is a thoroughly investigated accident involving the failure 
of a sensor (a set of Pitot tubes), resulting in incorrect 
speed readings and, through a sequence of events, to a high- 
altitude stall situation that failed to be diagnosed by the 


pilot(s). The BAE report on the accident 1 attributes the 
main cause of the accident to the inexperience of the pilot, 
who was not able to assess the actual speed of the airplane 
and, more crucially, the stall situation. 

We employ here a Java simulation model of the scenario 
taken from [1] and we modify it to generate a set of reach- 
able states using the approach presented in [16], in collab- 
oration with domain experts at NASA Ames. The set of 
reachable states obtained is then encoded in Mc-COGWED 
input. We remark that our model does not aim at being an 
accurate representation of the accident; instead, our aim is 
to show the capabilities of COGWED in analysing situation 
awareness. In our model, a plane and its environment are 
characterised by: 

• an actual external temperature (low, medium, high); 

• an actual speed (very low, low, medium, high, very 
high); 

• an actual vertical speed (Climbing, null, Descending); 

• an actual altitude (encoded using flight levels, such as 
FL200, FL380 and FL450); 

• an actual attitude (going up, flat, down); 

• an actual thrust level (auto, 20%, 50%, TOGA, full. 
“TOGA” is an auto-thrust level corresponding to the 
thrust required for Take-Off or a Go- Around landing) 

In the actual situation the pilot is flying in the dark: it 
only has direct access to the thrust, which we assume is 
correct. All the remaining parameters are accessed through 
sensors that may be faulty. As a result, we characterise the 
local states of the pilot by means of: 

• observed temperature; 

• observed speed; 

• observed vertical speed; 

• observed altitude; 

• observed attitude. 

All these these values are observed by means of sensors, 
some of which may fail. When a sensor is broken, the ob- 
served value of a parameter may differ from the actual value. 
Additionally, a plane includes: 

• an auto pilot to which the pilot has direct access, i.e., 
the pilot can observe whether the auto pilot is engaged 
or not, and we assume that the auto pilot does not fail. 

• a set of Pitot tubes that may be frozen when the tem- 
perature is low (but not necessarily). If the Pitot 
tubes are frozen, then the speed sensor is broken (but 
the speed sensor could be broken even when the Pitot 
tubes are not frozen). 

x http : / /www . bea . aero/ en/ enquetes/ 
flight . af . 447/flight . af . 447 . php 


• a stall warning (in the form of audio message or stick 
shaking, depending on the causes of the stall). Notice 
that the stall warning disengages when the speed is very 
low (below 60 kt), even if the plane could be actually 
stalling. We assume that the stall warning signal does 
not fail, i.e. a warning always corresponds to stalling 
conditions. 

We model the behaviour of the pilot based on the pro- 
cedures required in the various cases. For instance, if the 
observed speed is very high (a potentially very dangerous 
situation) the pilot reduces thrusts, and if the stall warning 
is on, the pilot modifies attitude and thrust appropriately. 
The Java simulation modifies the actual values of the air- 
plane characteristics according to pilot’s actions and stan- 
dard physics laws, generating new states every time a value 
changes. 

To generate the set of possible states for this scenario, we 
start from a situation in which the plane is flying at flight 
level 380 (corresponding to 38,000 feet), the thrust is 60%, 
the auto pilot is engaged, the stall warning is off, attitude is 
flat, temperature is medium and all sensors are working cor- 
rectly. We then inject failures in the sensors (as mentioned 
above, these are not diagnosable by the pilot: remember 
that the pilot is flying in the dark and has therefore no pos- 
sibility of assessing (vertical) speed, attitude, altitude and 
temperature) and we generate a COGWED model covering 
all possible combinations reachable from the initial state. 
The generation is achieved by running the Java code de- 
veloped in [1] and by discretizing the continuous variables 
where required (in this case: speed, vertical speed, attitude, 
altitude, temperature). The number of possible discretized 
states is 2- 10 s , of which approximately 1.6 TO 5 are reachable 
from the initial state described above. 

We can now use Mc-COGWED to evaluate the fact that 
the pilot is aware of a stall. In particular, we want to assess 
the degree of belief of a stall situation. To this end, we 
employ the following formula: 

J7E(actualStall a B<o°o 5 (actualStall)) 

This formula encodes the fact that there exists a state reach- 
able from the initial state, such that the plane in actually 
stalling, but in that specihc state the pilot believes that the 
stall is actually occurring with a degree of less than 5%: this 
formula is true in 25 states in the model. In fact, we can 
check that there are 5 stalling states in which the pilot be- 
lieves in a stall with a degree of less than 1.5%. These are 
very interesting configurations that capture what may have 
happened on board of AF447: in these 5 states, the speed 
sensor is faulty (as a result of the Pitot tubes being frozen) 
and may report wrong measures, the attitude is UP, the 
speed is very low, and as a result of this low speed the stall 
warning remains silent. Notice that, in these specihc cases, 
modifying the attitude to descend results in an increase in 
speed of the airplane, therefore re-starting the stall warning 
in the cabin: this is even more confusing for the pilot, as 
a manoeuvre that reduces the likelihood of stalling in fact 
generate a stall warning! 

The generation of all the discretized states and its encod- 
ing as a Mc-COGWED input Hie require less than a minute, 
and Mc-COGWED can verify the formula encoding situa- 
tional awareness for the stall situation in less than 8 seconds. 

We argue that the doxastic pattern above can be used to 
characterise (the lack of) situational awareness in the general 


case: the formula 

ip -> B' <s ip 

is true in states in which ip holds, but agent i has a degree of 
belief less than S that this is indeed the case. The parameter 
<5 could be configured depending on the specific domain, and 
can be interpreted as a measure of situational awareness. 

In the AF447 scenario, it is interesting to see how the 
situational awareness of a stall could be increased. The dis- 
engagement of the stall warning at low speed is justified by 
the necessity of performing low-speed operations close to the 
ground and to avoid spurious warnings, for instance when 
taking off or while landing; this, however, results in the pi- 
lot not being able to diagnose a stall at very low speed in 
other conditions. To address this issue, an additional visual 
indicator of stall warning with low speed readings could be 
added to the cockpit: this would be similar to ABS warn- 
ings on certain car models that remain active under 10 MPH. 
The additional indicator would reduce the number of possi- 
ble worlds that the pilot considers possible, thereby increas- 
ing the minimum value of 5 for which the formula above 
is true. This is exactly in line with the recommendations 
of the BAE to modify the stall management procedures on 
Airbuses, by re-designing the Primary Flight Display out- 
put and by adding additional training requirements in high- 
altitude stalling conditions. 

6. RELATED WORK 

Formalisms to model degrees of belief have been investi- 
gated in the past by a number of authors. Dempster-Shafer 
belief functions [22] are among the most common approaches 
to assign a mass to beliefs and to combine belief functions. 
This formalism is a classical example of subjective assign- 
ment in which plausibility can be modelled differently from 
probability. Due to space limitations, we refer to [13] for 
other approaches to modelling degrees of belief subjectively. 
In all these formalisms, however, the function associating a 
weight to a belief needs to be externally provided, for in- 
stance by employing historical data or other means; this is 
a key difference with our approach, where degrees are com- 
puted as the ratio between two sets of possible worlds. 

The idea of evaluating degrees of belief as the ratio be- 
tween possible worlds is not new: in the formalism of ran- 
dom worlds [2] degrees of belief are computed using propor- 
tion expressions of the form | \<p(x)\ip(x )\ . These expressions 
denote the the proportion of domain elements satisfying ip 
w.r.t. those satisfying ip in the domain of a knowledge base. 
Conditional expressions are used in [2] to evaluate the weight 
of beliefs in knowledge bases and are shown to satisfy a set 
of desiderata for default reasoning. While “computation- 
ally grounded” in the sense that degrees of belief are not 
provided externally, computing degrees of belief using ran- 
dom worlds is an undecidable problem in the general case. 
Moreover, there does not seem to be a tractable solution to 
add temporal reasoning to this formalism as we do here (as 
exemplified in the case of the dining cryptographers). Addi- 
tionally, another key difference with our approach is that we 
provide a formal language to express degrees of belief for a 
system of agents and we are not limited to the single agent 
case. Along similar lines, the work in [11] introduces plau- 
sibility measures that are used to justify a set of axioms for 
default reasoning. More recently, the work in [14] addresses 
decision making in terms of weighted sets of probabilities 


by introducing an axiomatization and by providing dynamic 
decision making procedures. 

A language that combines first-order logic and probabil- 
ity in finite domains is introduced in [21] using Markov 
Logic Networks (MLN): similarly to [2], knowledge bases 
are employed as the underlying semantics, and weights are 
associated to formulae in the KB. In the case of finite do- 
mains weights can be learned using a set of algorithms and 
the authors show that MLN can tackle real scenarios. The 
work in [7] presents the logic PpKD45, whose syntax is very 
similar to COGWED. The semantics of this logic relies on 
externally-provided probability measures over finite bases; 
the authors present an axiomatization and a decision proce- 
dure for this logic but no model checking algorithm. The key 
differences with our work are the different semantics based 
on interpreted systems and the inclusion of multiple agents 
and temporal modalities, in addition to a dedicated model 
checking tool. 

In the multi-agent system community there have been 
a number of works addressing the verification of doxastic 
modalities, such as the Jason tool [3] and the AIL+AJPF 
framework [8]. These two works address BDI architectures 
and are capable of verifying “standard” (i.e., non- weighted) 
doxastic operators. The tool MCK [12] has recently been ex- 
tended to include probabilistic reasoning. In this tool prob- 
abilities are assigned to temporal relations; the tool is able 
to verify only the probability of Boolean expressions, possi- 
bly nested in an X (next-state) temporal operator. Proba- 
bilities over temporal relations are also analysed using the 
logic PCTL in the well known tool PRISM [17], which has 
recently been extended to verify probabilistic ATL [5]. A 
logic to reason about probabilistic knowledge and strategies 
is also described in [15]: in this work probabilities are associ- 
ated to temporal relations and to observations as well. Our 
key difference is again in the definition of degrees of belief 
in terms of possible worlds. 

More importantly, the PRISM and MCK tool and the ap- 
proach in [15] all employ probabilities over transitions. As 
mentioned in the introduction, we refer instead to degrees 
of belief. The relationship between these two concepts has 
been investigated in [2] for a scenario very similar to ours, 
where degrees are computed as the ratio between two sets 
of possible worlds. Similarly to this work, in our setting all 
the possible worlds are equally likely and we do not model 
probabilities of transitions. Essentially, our approach adopts 
the principle of indifference by Bernoulli and Laplace. As 
described in [2], a uniform distribution for possible worlds is 
the one that maximizes entropy. In turn, this corresponds to 
the least amount of information about the probability dis- 
tribution of epistemically equivalent worlds. In other words, 
we start from an unknown objective assignment of probabil- 
ities to transitions and we build a subjective assignment of 
degrees of belief to agents according to this unknown objec- 
tive assignment; agents’ degrees of belief can then be inter- 
preted using a computationally grounded evaluation. 

7. CONCLUSION 

In this paper we have introduced COGWED, an extension 
of CTLK to reason about degrees of belief in a system of 
agents. We have introduced a computationally grounded se- 
mantics based on Interpreted Systems, we have presented a 
model checking algorithm for COGWED and we have inves- 
tigated its complexity, showing that model checking COG- 


WED has the same complexity of model checking CTLK. To 
validate our claims, we have implemented and made publicly 
available Mc-COGWED, a Java-based explicit state model 
checker, and we have assessed the performance of our algo- 
rithm against the standard benchmark of the dining cryp- 
tographers. The results obtained are very encouraging: our 
prototype was able to verify up to 15 cryptographers, a fig- 
ure comparable to state-of-the-art model checkers for multi- 
agent systems. 

To prove the applicability of COGWED to real scenarios 
we have collaborated with domains experts at NASA Ames 
to assess the situational awareness of aircraft pilots flying 
in off-nominal conditions, obtaining results that are in line 
with BAE recommendations. Finally, we have presented 
a detailed review of related work, highlighting our contri- 
butions and discussing the relationship between degrees of 
belief as modelled in COGWED and probability measures 
over temporal transitions. 

As mentioned in the previous section, our approach con- 
siders all the possible worlds equally likely: this is the result 
of ignoring the probability distribution of temporal transi- 
tions. We are currently working at incorporating this in- 
formation into the doxastic characterisation of agents. In 
particular: what can be said when the pilot knows that a cer- 
tain sensor has a higher probability of failure than another 
sensor? What could be said about the resulting degrees of 
belief? 
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